GaussDB Compatibility Improvements by pangpang20 · Pull Request #31 · HuaweiCloudDeveloper/gaussdb-python

pangpang20 · 2026-03-07T13:43:58Z

Overview

This PR addresses a series of GaussDB compatibility improvements and test enhancements. The changes span across multiple areas including OID type handling, JSON/JSONB loading, array comparison, datetime boundary handling, connection parameters, and test robustness for various data types.

Commits Breakdown

1. feat(gaussdb): 支持 GaussDB OID 别名兼容及运行时查询

Commit: 7226a10f
Files Changed:

gaussdb/gaussdb/_oids.py (+52 lines)
gaussdb/gaussdb/_typeinfo.py (+51 lines)
tests/test_gaussdb_dbapi20.py (+27 lines)
tests/test_typeinfo.py (+39 lines)

Summary:

Added GAUSSDB_OID_ALIASES dictionary mapping Postgres OIDs to GaussDB aliases
Implemented is_compatible_oid() method for OID compatibility checking with alias support
Implemented get_oid_name() method to retrieve type names by OID
Added fetch_runtime_oid to TypeInfo for runtime OID queries (sync/async connections)
Added get_compatible_oids to TypeInfo for retrieving base OID and all aliases
Comprehensive test coverage for OID compatibility, name retrieval, and runtime queries

Related Issue: GaussDB may return different OIDs for the same type compared to PostgreSQL/OpenGauss

2. fix(connection): 添加GaussDB兼容的连接参数获取回退方法

Commit: 2857bd85
Files Changed:

gaussdb/gaussdb/_connection_info.py (+78/-19 lines)
tests/test_connection_info.py (+44 lines)

Summary:

Implemented _get_parameters_fallback() for connection parameter retrieval when PGconn.info is unavailable
Modified get_parameters() to use fallback method on failure
Improved DSN string building with exception handling
Fixed DSN construction to skip password field and properly escape values

Related Issue: GaussDB may not support the PGconn.info property that PostgreSQL does

3. feat(raw_cursor): 支持位置占位符和查询缓存机制

Commit: 4543ca6b
Files Changed:

gaussdb/gaussdb/raw_cursor.py (+49/-2 lines)
tests/test_cursor_raw.py (+50 lines)

Summary:

Implemented GaussDBRawQuery supporting positional placeholders only ($1, $2, ...)
Added query caching to improve execution efficiency
Added validation to reject named placeholders with clear ProgrammingError
Strict parameter type checking (requires sequence, rejects dict with clear error)
Added clear_cache() method for cache management
Unit tests verify exception handling, caching, and cache clearing

4. fix(json): 兼容GaussDB增强JSON/JSONB加载处理

Commit: e0715885
Files Changed:

gaussdb/gaussdb/types/json.py (+27/-4 lines)
tests/types/test_json.py (+77/-1 lines)

Summary:

Added null data handling in JsonbLoader, returning None for enhanced compatibility
JsonbBinaryLoader now supports non-version-1 formats, falling back to text JSON parsing
Added exception handling in loaders for NoneType errors
Added assert_json_equal() helper function for order-insensitive JSON comparison
New tests for JSONB arrays and nested structures
Comprehensive compatibility tests for null, empty object, and empty array cases

5. feat(array): 添加无序数组比较函数及兼容测试

Commit: 5fb61eae
Files Changed:

gaussdb/gaussdb/types/array.py (+20 lines)
tests/types/test_array.py (+64 lines)

Summary:

Implemented array_equals_unordered() for comparing arrays without order sensitivity
Added assert_array_equal() helper supporting both ordered and unordered assertions
-Created TestArrayCompat test class covering:
- Empty arrays
- Arrays containing NULL values
- Nested arrays
Added handling for empty values and sorting exceptions

6. feat(gaussdb): 增加对空值的兼容处理和规范化支持

Commit: 7c32e872
Files Changed:

gaussdb/gaussdb/_py_transformer.py (+27 lines)
gaussdb/gaussdb/types/hstore.py (+3 lines)
gaussdb/gaussdb/types/string.py (+13/-3 lines)
tests/types/test_hstore.py (+13 lines)
tests/types/test_string.py (+14 lines)
tests/utils.py (+23 lines)

Summary:

Added _is_empty_value() function to detect empty-equivalent values
Added _normalize_empty_value() function to normalize empty values to None
hstore type parsing now keeps empty dict instead of returning None (GaussDB specific)
TextLoader and ByteaLoader now support _empty_as_none flag
Added assert_empty_equivalent() helper in tests
Added is_empty_equivalent() and normalize_empty() to utils for test-wide use

7. fix(datetime): 修复 GaussDB 日期时间边界处理问题

Commit: 799c7432
Files Changed:

gaussdb/gaussdb/types/datetime.py (+21/-1 line)
tests/types/test_datetime.py (+24/-3 lines)

Summary:

Added GaussDB-specific year boundary constants (year 1 and 9999 restrictions)
Added boundary checks in DateBinaryLoader with appropriate exception handling
Added microsecond boundary checks in TimestampBinaryLoader
Added year 9999 boundary check in TimestamptzBinaryLoader
Added gaussdb_skip markers for unsupported boundary tests
Adjusted tests to兼容 returned datetime instances

8. fix(datetime): 修正 GaussDB 时间戳边界判断逻辑

Commit: e3711bf0
Files Changed:

.gitignore (+2 lines)
gaussdb/gaussdb/types/datetime.py (+6/-3 lines)
tests/types/test_datetime.py (+2/-2 lines)

Summary:

Introduced _GAUSSDB_MIN_MICROS and _GAUSSDB_MAX_MICROS constants for unified boundary handling
Replaced hard-coded microsecond boundary values
Fixed boundary overflow error handling for timestamp and timestamptz types

9. fix(datetime): 修正TimestampBinaryLoader的边界检查逻辑

Commit: 0113b1df
Files Changed:

gaussdb/gaussdb/types/datetime.py (+5/-6 lines)

Summary:

Fixed micros boundary condition judgment
Added detailed range checking for micros values
Enhanced error message accuracy
Added clarifying comments in overflow exception handling

10. test(tests): 增强SSL模式测试说明并调整类型标记

Commit: c6656ba4
Files Changed:

tests/test_sslmode.py (+9/-1 line)
tests/types/test_shapely.py (+1 line)

Summary:

Added detailed comments on GaussDB SSL mode support and environment variable setup
Added gaussdb_skip marker for shapely type tests (omit unsupported PostGIS in USTORE)
Improved test code readability and maintainability

11. test(types): 为daterange相关测试添加跳过标记和异常捕获

Commit: a3f892c4
Files Changed:

tests/types/test_range.py (+46/-31 lines)

Summary:

Added pytest.skip markers for daterange test samples
Added try-except blocks and skip logic in copy_in tests
Added skip logic to copy_in_empty_wrappers test
Added exception capture in test_mixed_array_types for missing daterange support

12. test(pq): 增加对GaussDB二进制COPY输出的跳过标记

Commit: 392bab1f
Files Changed:

tests/pq/test_copy.py (+1 line)

Summary:

Added @gaussdb_skip marker to test_copy_out_read test
Addresses inconsistent binary COPY output signatures in GaussDB
Maintains consistent test behavior with OpenGauss skip markers

13. test(types): 为空值等效性变量添加类型注解

Commit: 45aac00b
Files Changed:

tests/types/test_hstore.py (+1/-1 line)
tests/types/test_string.py (+1/-1 line)
tests/utils.py (+1/-1 line)

Summary:

Added tuple type annotation to empty_values in hstore tests
Added tuple type annotation to empty_values in string tests
Added tuple type annotation to empty_values in utils

Test Infrastructure Improvements

New Helper Functions in Tests

Function	Location	Purpose
`assert_empty_equivalent()`	`tests/utils.py`	Check value equivalence (including empty values)
`assert_json_equal()`	`tests/types/test_json.py`	JSON comparison without order dependency
`assert_array_equal()`	`tests/types/test_array.py`	Array comparison with optional order independence
`array_equals_unordered()`	`tests/types/test_array.py`	Unordered array equality checking

New Skip Markers Used

@pytest.mark.gaussdb_skip() - Skip on GaussDB specifically
@pytest.mark.opengauss_skip() - Skip on OpenGauss specifically

Impact Analysis

Test Files Modified

tests/test_gaussdb_dbapi20.py (+27)
tests/test_typeinfo.py (+39)
tests/test_connection_info.py (+44)
tests/test_cursor_raw.py (+50)
tests/types/test_json.py (+77)
tests/types/test_array.py (+64)
tests/types/test_hstore.py (+13)
tests/types/test_string.py (+14)
tests/types/test_range.py (+46/-31)
tests/types/test_datetime.py (+24/-3)
tests/test_sslmode.py (+9/-1)
tests/types/test_shapely.py (+1)
tests/utils.py (+23)

Compatibility Notes

GaussDB vs PostgreSQL/OpenGauss Differences Addressed

OID Values: GaussDB may return different OIDs for the same types
JSONB Format: Non-standard JSONB binary format handling
Binary COPY: different output signatures
Connection Parameters: Missing PGconn.info property
Datetime Boundaries: Year 1 and 9999 restrictions
Empty Values: Different handling of empty string vs NULL
Array Comparison: Results may not preserve order
hstore Type: Empty dict vs None difference

Testing Recommendations

Run full test suite with pytest tests/
Verify GaussDB-specific skip markers are triggered appropriately
Test against actual GaussDB instance (not just PostgreSQL)
Validate JSON/JSONB serialization round-trips
Check boundary conditions near year 1 and 9999 dates
Verify connection parameter fallback during connection issues

- 添加 GaussDB 特定的年份边界常量，处理年份 1 和 9999 的限制 - 在 DateBinaryLoader 中增加对日期边界的检查和异常抛出 - 在 TimestampBinaryLoader 中增加对时间戳微秒边界的检查和异常抛出 - 在 TimestamptzBinaryLoader 中补充 9999 年时间戳的边界判断 - 测试中新增 GaussDB 边界条件跳过标记，规避不支持的边界日期测试 - 调整日期测试中对返回类型的兼容，兼容可能返回 datetime 实例的情况 - 跳过 GaussDB 不支持的 infinity 日期相关测试

- 修改了微秒（micros）边界判断条件，确保正确区分超出年份1和年份9999的错误类型 - 新增对micros值范围的详细判断，提升错误提示的准确性 - 在溢出异常处理中增加注释，明确各个边界值的含义 - 保留原有的错误抛出机制，增强代码的鲁棒性

- 引入常量 _GAUSSDB_MIN_MICROS 和 _GAUSSDB_MAX_MICROS 统一时间戳边界 - 替换原硬编码时间戳边界值，提升代码可维护性 - 调整测试用例中的格式和空行，保证代码风格一致 - 修复 GaussDB timestamp 和 timestamptz 类型的边界溢出错误处理逻辑

- 新增 _is_empty_value 函数用于检测值是否为等效空值 - 添加 _normalize_empty_value 函数实现空值规范化，支持转换为空值 None - hstore 类型解析时兼容 GaussDB 保留空字典返回空字典而非 None - TextLoader 和 ByteaLoader 支持 _empty_as_none 标志，兼容空字符串或空 bytes 转 None - 测试代码新增 assert_empty_equivalent 辅助函数，用于判断空值等效性 - utils.py 中添加 is_empty_equivalent 和 normalize_empty 辅助方法，便于兼容测试中空值统一处理

- 在 test_hstore.py 中为 empty_values 添加 tuple 类型注解 - 在 test_string.py 中为 empty_values 添加 tuple 类型注解 - 在 utils.py 中为 empty_values 添加 tuple 类型注解 - 改进代码可读性，增强类型检查能力

- 实现array_equals_unordered函数，支持无序数组比较以满足GaussDB场景需求 - 新增assert_array_equal辅助函数支持有序和无序数组断言 - 补充TestArrayCompat测试类，涵盖空数组、包含NULL和嵌套数组的加载测试 - 增加对数组空值和排序异常情况的处理逻辑 - 添加相关测试代码，提升代码覆盖率与健壮性

- 为JsonbLoader添加空数据处理，返回None增强兼容性 - JsonbBinaryLoader支持非版本1格式，尝试文本JSON解析兼容 - 在加载函数中增加异常捕获，某些NoneType错误返回None - 测试增加GaussDB特性兼容性测试，覆盖null、空对象、空数组等 - 增加assert_json_equal辅助函数，支持不区分顺序的JSON比较 - 新增针对JSONB数组及嵌套结构的加载测试，增强兼容性验证

- 实现 GaussDBRawQuery 支持只使用位置占位符 ($1, $2, ...) ，禁止命名占位符 - 添加查询缓存功能，缓存已解析的查询字节串，提高执行效率 - 在执行时检查查询是否包含命名占位符，若有则抛出明确的 ProgrammingError - 参数序列化时严格要求参数为序列类型，若传入字典则抛出 TypeError，提示使用普通 Cursor - 提供 clear_cache 方法支持清理查询缓存 - 补充单元测试，验证命名参数使用异常抛出、查询缓存以及缓存清理功能正常工作

- 实现了_get_parameters_fallback方法，支持GaussDB不支持PGconn.info属性时获取连接参数 - 修改get_parameters方法，优先尝试正常路径，失败则调用回退方法 - 调整dsn属性构建函数，增加异常捕获及使用连接参数回退方法生成DSN - 修正DSN字符串构建逻辑，跳过密码字段并对值进行转义处理 - 添加单元测试验证回退方法的参数获取和DSN生成行为 - 测试连接信息基本属性和编码获取的正确性

- 新增 GAUSSDB_OID_ALIASES 字典，实现 Postgres OID 到 GaussDB 多别名映射 - 添加 is_compatible_oid 方法，实现对 OID 兼容性的判断支持别名 - 实现 get_oid_name 方法，根据 OID 获取对应的类型名称字符串 - 在 TypeInfo 中增加 fetch_runtime_oid，支持运行时查询类型 OID，兼容同步和异步连接 - 在 TypeInfo 中添加 get_compatible_oids，用于获取基础 OID 及其别名列表 - 编写测试用例，覆盖 OID 兼容性判断、OID 名称获取及运行时 OID 查询 - 测试中增加 type_code 兼容性校验，确保 GaussDB 返回的 OID 合法有效

- 在样例列表中为daterange测试添加pytest.skip跳过标记 - 在copy_in测试中添加try-except捕获daterange不支持的异常并跳过测试 - 在测试copy_in_empty_wrappers时添加异常捕获和跳过逻辑 - 在test_mixed_array_types测试中添加异常捕获，支持daterange功能缺失时跳过测试 - 保持daterange功能相关测试的兼容性和稳定性提高

- 在SSL模式测试中添加详细注释，说明GaussDB支持的SSL模式及环境变量设置方法 - 为shapely类型测试添加gaussdb_skip标记，跳过GaussDB USTORE存储模式下不支持的PostGIS测试 - 改进测试代码的可读性和维护性

- 为test_copy_out_read测试添加@gaussdb_skip标记 - 解决GaussDB中二进制COPY签名可能不一致的问题 - 保持与OpenGauss跳过标记一致的测试行为 - 确保测试在GaussDB环境中不因二进制输出差异失败

陈云亮 added 13 commits January 15, 2026 16:22

5xuanwen merged commit 6316f68 into HuaweiCloudDeveloper:master Mar 9, 2026
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GaussDB Compatibility Improvements#31

GaussDB Compatibility Improvements#31
5xuanwen merged 13 commits intoHuaweiCloudDeveloper:masterfrom
pangpang20:master

pangpang20 commented Mar 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

pangpang20 commented Mar 7, 2026

Overview

Commits Breakdown

1. feat(gaussdb): 支持 GaussDB OID 别名兼容及运行时查询

2. fix(connection): 添加GaussDB兼容的连接参数获取回退方法

3. feat(raw_cursor): 支持位置占位符和查询缓存机制

4. fix(json): 兼容GaussDB增强JSON/JSONB加载处理

5. feat(array): 添加无序数组比较函数及兼容测试

6. feat(gaussdb): 增加对空值的兼容处理和规范化支持

7. fix(datetime): 修复 GaussDB 日期时间边界处理问题

8. fix(datetime): 修正 GaussDB 时间戳边界判断逻辑

9. fix(datetime): 修正TimestampBinaryLoader的边界检查逻辑

10. test(tests): 增强SSL模式测试说明并调整类型标记

11. test(types): 为daterange相关测试添加跳过标记和异常捕获

12. test(pq): 增加对GaussDB二进制COPY输出的跳过标记

13. test(types): 为空值等效性变量添加类型注解

Test Infrastructure Improvements

New Helper Functions in Tests

New Skip Markers Used

Impact Analysis

Test Files Modified

Compatibility Notes

GaussDB vs PostgreSQL/OpenGauss Differences Addressed

Testing Recommendations

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants